1. Field of Invention
The invention pertains to the fields of pattern recognition, scene segmentation, machine vision and machine speech recognition, and neuromorphic engineering. The invention constitutes a basic building block of a new class of systems to implement perceptual and cognitive functions in various sensory and conceptual domains.
2. Discussion of Related Art
The history of neurologically inspired computing technology goes back at least several decades. The field lay relatively quiescent until the mid 1980s when the popularization of the multi-layer xe2x80x9cartificial neural netxe2x80x9d (ANN), usually emulated in software, overcame a decade and a half of skepticism about the technology largely engendered by an influential book xe2x80x9cPerceptronsxe2x80x9d by Marvin Minsky and Seymour Papaert. The multilayer ANN, particularly ones trained by various algorithms of backward propagation of an error signal, proved capable of a wide range of classification tasks. Though the differences between ANNs and biological neural circuitry was evident, the lack of an alternative computational hypothesis for the biological circuitry at first attracted some in the neuroscience community to adopt the ANN in its various forms as a model of neural computation. The limitations of the ANN, however, became apparent in both technological application and as a computational model for the brain. The ANN has become an accepted part of the computational repertoire for solving certain classes of classification problems and problems involving capturing of complex functions from a large set of training data.
However, the ANN on its own has not provided the basis for solutions to the more complex problems of vision, speech recognition, and other sensory and cognitive computation. Problems of segmentation, recognition of signals under various transformations, learning from single or very limited presentations of data, regularity extract, and many other real world recognition problem have to date eluded solution by ANN or any other neurally inspired techniques.
Experimental neuroscience has revealed that cortical and other neural architecture is far more complicated than any ANN. Realistic simulations of large neurons such as pyramidal or Purkinje cells suggest that individual neurons are capable of significant computation on their own, which must be the basis of the computations performed by circuits containing myriads of such cells. Experiment has also determined that connectivity between neurons is highly specific and becomes more so during the initial learning by organisms. Neurons themselves are specialized, not uniform. Strong evidence exists that synchronized oscillations across the cortices play a role in recognition. And perhaps most important is the fact that backward or descending signal paths are at least as numerous as the forward or ascending signal paths from sensory organs to progressively xe2x80x9chigherxe2x80x9d cortices.
The invention (the e-cell and building block e-circuit modules composed of e-cells) constitutes the base components of a technology system inspired by the neuroanatomical and neurophysiological evidence mentioned above which is ignored by the traditional ANN technology, and gives rise to system organization and computational capabilities quite distinct from ANNs. When combined into basic e-circuit modules, the e-cell provides the basis of systems capable of scene segmentation, recognition of signals under translation, and where applicable under scaling and rotation and well as degradation by noise and occlusion. The systems are capable of learning novel percepts with a single presentation, establishing association and context between percepts and percept classes. In addition the systems are capable, without supervision or auxiliary xe2x80x9chintsxe2x80x9d, of extracting signal regularities from unsegmented signal fields so as to identify which segments of the fields constitute stable percepts such as words or objects.
The E-Cell
The e-cell is the sole computational element of which circuits are built. It can be configured by a combination of specification and learning to perform a multiplicity of computational tasks. Important configuration variants involve
(1) xe2x80x9cdendriticxe2x80x9d architecture and behavior
(2) rise time and decay rates
(3) learning protocols
(4) bias inputs
It consists of a cell body, an output terminal, and set of tree-connected input regions each of which have one or more input terminals and corresponding stored weights. Each input and its corresponding weight is termed a synapse. Each region computes the dot product of the vector of its current inputs and current weights and may modify this with a bias term. The regions may be instructed to learn by altering their current weights as a function of the currently applied inputs to each region. The regions are joined to the cell body by a binary tree structure in which the outputs of the regions comprise the leaves of the tree and each node is a two input junction which applies one of two classes of function to the node""s pair of inputs, each of which may be either a region output or the output of another node. The body of the cell has at least one inputxe2x80x94the output of the root node of the treexe2x80x94and may have a bias input and a learning control input. The body of the cell computes the new output value of the cell based on the output of the root node of the tree and the current output of the cell itself. If the cell has a bias input, the value of that bias input is distributed to a certain class of regions which modify the dot product of their input and weight vectors by either adding the bias or by multiplying by 1.0 plus bias. If the cell has a learning input, the value of that input is added to the output of the root node and if a certain learning threshold is exceeded then all the regions capable of learning are signaled to apply their learning algorithm. If the sum of the inputs to a region instructed to learn are either zero or below a certain low threshold, the region follows one of two rules: (1) its weights are set to zero or (2) its weights are left unchanged. If the sum of inputs to a region instructed to learn are above the mentioned threshold, then the region alters its weight in one of two ways: (1) each weight is altered to be proportional to its corresponding input such that the dot product of the new weights and current inputs is normalized to a standard output (e.g. 1.0) or (2) each weight is altered incrementally from its current weight toward the weights that would satisfy condition (1). The first learning rule is termed episodic learning and is intended to capture the values of a set of inputs in one learning cycle. The second learning rule is termed incremental and is intended to capture a set of values representative of multiple but related sets of inputs.
Basic E-Circuit Modules
The invention includes a set of elementary e-circuit modules assembled from e-cells. The resulting system organization and computational capabilities are quite distinct from ANNs. The e-circuit modules constituting this invention can be composed into higher level e-circuits capable of scene segmentation, recognition of signals under translation, and where applicable under scaling and rotation and well as degradation by noise and occlusion. The systems are capable of learning novel percepts with a single presentation, establishing association and context between percepts and percept classes. These higher level e-circuits are the subject of an associated utility application.
The first component e-circuit of the invention is termed the basic e-cell pair. It actually contains three e-cells. The two major e-cells (the pair) are connected in a xe2x80x9ccomplementaryxe2x80x9d fashion thus providing segments of associated forward and backward signal paths. An inhibitory e-cell connects the output of backward e-cell to an input of the forward e-cell and thus provides a inhibitory feedback path which is responsible for the oscillatory dynamics at the heart of the e-circuit""s computation. The inhibitory e-cell has no connection external to the basic e-cell pair module. The basic e-cell pair also provides an excitatory connection between the output of the forward e-cell to an input of the backward e-cell which can provided the closure of the loop from forward to backward paths.
Populous groups of e-cell pairs comprise the stages of e-circuits. The e-cell populations that make up stages are termed groups and are divided into clusters.
E-circuits always consist of multiple stages, so any e-cell pair will be connected to one or more e-cell pairs upstream and/or downstream of it. A sequence of two or more interconnected e-cell pairs from adjacent stages constitutes the basic e-cell pair totem. The totem is a useful unit to analyze in isolation because it determines the fundamental dynamics of the e-circuit. In action, e-circuits behave to a large extent as large collections of totems, though the totems are not independent due to lateral connections between them and convergences of a number of totems on a single e-cell pair in certain stages of the circuit.
The basic two-stage totem provides the structure which is specialized to become the basic memory circuit, which is the second standard e-circuit component of this invention.
The basic memory circuit is extended to a third stage to become the basic memory circuit with context and association, which is the third standard e-circuit component of this invention.